Brain-inspired Recurrent Neural Algorithms for Advanced Object Recognition

Deep learning has enabled breakthroughs in machine learning, resulting in performance levels seeming on par with humans. This is particularly the case in computer vision where models can learn to classify objects in the images of a dataset. These datasets however often feature perfect information, whereas objects in the real world are frequently cluttered with other objects and thereby occluded. In this thesis, we argue for the usefulness of recurrency for solving these questions of partial information and visual context. First, we show that humans robustly recognize partial objects even at low visibilities while today’s feed-forward models in computer vision are not robust to occlusion with classification performance lacking far behind human baseline. We argue that recurrent computations in the visual cortex are the crucial piece, evident through performance deficits with backward masking and neurophysiological delays for partial images. By extending the neural network Alexnet with recurrent connections at the last feature layer, we are able to outperform feed-forward models and even human subjects. These recurrent models also correlate with human behavior and capture the effects of backward masking. Second, we empirically demonstrate that human subjects benefit from visual context in recognizing difficult images. Building on top of feed-forward Alexnet, we add scene information and recurrent connections to the object prediction layer to define a simple model capable of context integration. Through the use of semantic relationships between objects and scenes derived from a lexical corpus, we can define the recurrent weights without training on large image datasets. The results of this work suggest that recurrent connections are a powerful tool for integrating spatiotemporal information, allowing for the robust recognition of even complex images.